Supervised learning aims to train a classifier under the assumption that training and test data are from the same distribution. To ease the above assumption, researchers have studied a more realistic setting: out-of-distribution (OOD) detection, where test data may come from classes that are unknown during training (i.e., OOD data). Due to the unavailability and diversity of OOD data, good generalization ability is crucial for effective OOD detection algorithms. To study the generalization of OOD detection, in this paper, we investigate the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we also offer theoretical supports for several representative OOD detection works based on our OOD theory.

循环结束是自动移动系统同时本地化和映射（SLAM）的基本组成部分。在视觉大满贯领域，单词袋（弓）在循环封闭方面取得了巨大的成功。循环搜索的弓特征也可以在随后的6-DOF环校正中使用。但是，对于3D激光雷达的猛击，最新方法可能无法实时识别循环，并且通常无法纠正完整的6-DOF回路姿势。为了解决这一限制，我们呈现了一袋新颖的单词，以实时循环在3D LIDAR大满贯中关闭，称为Bow3D。我们方法的新颖性在于，它不仅有效地识别了重新审视的环路，而且还实时纠正了完整的6型循环姿势。 BOW3D根据3D功能link3D构建单词袋，该链接有效，姿势不变，可用于准确的点对点匹配。我们将我们提出的方法嵌入了3D激光射击系统中，以评估循环闭合性能。我们在公共数据集上测试我们的方法，并将其与其他最先进的算法进行比较。在大多数情况下，BOW3D在F1 MAX和扩展精度分数方面表现出更好的性能，并具有出色的实时性能。值得注意的是，BOW3D平均需要50毫秒才能识别和纠正Kitti 00中的循环（包括4K+ 64射线激光扫描），当在使用Intel Core i7 @2.2 GHz处理器的笔记本上执行时。

及时调整尝试更新预训练模型中的一些特定任务参数。它的性能与在语言理解和发电任务上的完整参数设置的微调相当。在这项工作中，我们研究了迅速调整神经文本检索器的问题。我们引入参数效率的及时调整，以调整跨内域，跨域和跨主题设置的文本检索。通过广泛的分析，我们表明该策略可以通过基于微调的检索方法来减轻两个问题 - 参数 - 信息和弱推广性。值得注意的是，它可以显着改善检索模型的零零弹性概括。通过仅更新模型参数的0.1％，及时调整策略可以帮助检索模型获得比所有参数更新的传统方法更好的概括性能。最后，为了促进回猎犬的跨主题概括性的研究，我们策划并发布了一个学术检索数据集，其中包含18K查询的87个主题，使其成为迄今为止特定于特定于主题的主题。

特征提取和匹配是许多计算机视觉任务的基本部分，例如2D或3D对象检测，识别和注册。众所周知，2D功能提取和匹配已经取得了巨大的成功。不幸的是，在3D领域，由于描述性和效率低下，目前的方法无法支持3D激光雷达传感器在视觉任务中的广泛应用。为了解决此限制，我们提出了一种新颖的3D特征表示方法：3D激光点云的线性关键点表示，称为link3d。 Link3D的新颖性在于它完全考虑了LiDar Point Cloud的特征（例如稀疏性，场景的复杂性），并用其强大的邻居键盘来表示当前关键点，从而对当前关键点的描述提供了强烈的约束。提出的链接3D已在两个公共数据集（即Kitti，Steven VLP16）上进行了评估，实验结果表明，我们的方法在匹配性能方面的最先进表现都大大优于最先进的方法。更重要的是，Link3D显示出出色的实时性能（基于LIDAR的频率10 Hz）。 Link3D平均仅需32毫秒即可从64射线激光束收集的点云中提取功能，并且仅需大约8毫秒即可匹配两次LIDAR扫描，当时用Intel Core i7 @2.2 GHz处理器执行笔记本。此外，我们的方法可以广泛扩展到各种3D视觉应用。在本文中，我们已将Link3D应用于3D注册，LiDAR ODOMETIRE和放置识别任务，并与最先进的方法相比实现了竞争成果。

自我监督的学习表明它有可能在没有人为注释的情况下提取强大的视觉表现。提出各种作品从不同的角度处理自我监督的学习：（1）对比学习方法（例如，MOCO，SIMCLR）利用阳性和阴性样品来引导训练方向; （2）不对称网络方法（例如，BYOL，SIMSIAM）通过引入预测器网络和止动梯度操作来摆脱阴性样本; （3）特征去相关方法（例如，Barlow Twins，ViCREG），而是旨在降低特征尺寸之间的冗余。这些方法在各种动机的设计损失功能中看起来非常不同。最终的准确度数也各不相同，其中不同的网络和技巧在不同的作品中使用。在这项工作中，我们证明这些方法可以统一成相同的形式。我们不是比较他们的损失函数，我们通过梯度分析推出统一的公式。此外，我们进行公平和详细的实验以比较他们的表现。事实证明，这些方法之间几乎没有差距，并且使用动量编码器是提高性能的关键因素。从这个统一的框架来看，我们提出了一个简单但有效的自我监督学习的简单但有效的渐变形式。它不需要内存银行或预测的网络，但仍然可以实现最先进的性能，并轻松采用其他培训策略。广泛的线性评估实验和许多下游任务也表现出其有效性。代码应释放。

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.

Despite significant progress in object categorization, in recent years, a number of important challenges remain; mainly, the ability to learn from limited labeled data and to recognize object classes within large, potentially open, set of labels. Zero-shot learning is one way of addressing these challenges, but it has only been shown to work with limited sized class vocabularies and typically requires separation between supervised and unsupervised classes, allowing former to inform the latter but not vice versa. We propose the notion of vocabulary-informed learning to alleviate the above mentioned challenges and address problems of supervised, zero-shot, generalized zero-shot and open set recognition using a unified framework. Specifically, we propose a weighted maximum margin framework for semantic manifold-based recognition that incorporates distance constraints from (both supervised and unsupervised) vocabulary atoms. Distance constraints ensure that labeled samples are projected closer to their correct prototypes, in the embedding space, than to others. We illustrate that resulting model shows improvements in supervised, zero-shot, generalized zero-shot, and large open set recognition, with up to 310K class vocabulary on Animal with Attributes and ImageNet datasets.

Advances in computer vision and machine learning techniques have led to significant development in 2D and 3D human pose estimation from RGB cameras, LiDAR, and radars. However, human pose estimation from images is adversely affected by occlusion and lighting, which are common in many scenarios of interest. Radar and LiDAR technologies, on the other hand, need specialized hardware that is expensive and power-intensive. Furthermore, placing these sensors in non-public areas raises significant privacy concerns. To address these limitations, recent research has explored the use of WiFi antennas (1D sensors) for body segmentation and key-point body detection. This paper further expands on the use of the WiFi signal in combination with deep learning architectures, commonly used in computer vision, to estimate dense human pose correspondence. We developed a deep neural network that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions. The results of the study reveal that our model can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches, by utilizing WiFi signals as the only input. This paves the way for low-cost, broadly accessible, and privacy-preserving algorithms for human sensing.

With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few training examples. It has been a new trend exploring ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress, challenges, and future work in ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques of ICL, including training strategies, prompting strategies, and so on. Finally, we present the challenges of ICL and provide potential directions for further research. We hope our work can encourage more research on uncovering how ICL works and improving ICL in future work.

Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work focuses on the former. Previous methods build the network with several modules like CNN, LSTM and Attention. Recent methods combine the Transformer with these modules for better performance. However, it requires tedious optimization skills to train a network composed of mixed modules, making these methods inconvenient to be used in practice. In this paper, we propose to design \emph{pure Transformer-based networks} for deep RL, aiming at providing off-the-shelf backbones for both the online and offline settings. Specifically, the Transformer in Transformer (TIT) backbone is proposed, which cascades two Transformers in a very natural way: the inner one is used to process a single observation, while the outer one is responsible for processing the observation history; combining both is expected to extract spatial-temporal representations for good decision-making. Experiments show that TIT can achieve satisfactory performance in different settings, consistently.